Skip to content

feat(rag): add answer and KB quality evals#49

Merged
drewstone merged 1 commit into
mainfrom
feat/rag-quality-eval
Jul 7, 2026
Merged

feat(rag): add answer and KB quality evals#49
drewstone merged 1 commit into
mainfrom
feat/rag-quality-eval

Conversation

@drewstone

Copy link
Copy Markdown
Contributor

Summary

  • add general RAG answer eval primitives for context quality, answer support, citations, abstention, and failure diagnosis
  • add external score normalization and row exporters for Ragas, DeepEval, TruLens, RAGChecker, and custom evaluators
  • add generic KB quality scoring for citation coverage, source freshness, source-backed pages, duplicates, and lint/validation failures

Verification

  • pnpm exec biome check src tests
  • pnpm exec tsc --noEmit
  • pnpm exec vitest run tests/rag-eval.test.ts tests/rag-improvement-loop.test.ts tests/retrieval-eval.test.ts
  • pnpm exec tsup
  • pnpm exec vitest run

@tangletools tangletools left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

✅ Auto-approved drewstone PR — 1cd2cdd9

This PR was opened by the trusted drewstone account.
The full PR reviewer audit still runs separately and will publish findings if it detects issues.

tangletools · auto-approval · reason: drewstone_author · 2026-07-07T20:43:17Z

@drewstone drewstone merged commit 661c806 into main Jul 7, 2026
1 check passed
@drewstone drewstone deleted the feat/rag-quality-eval branch July 7, 2026 20:46
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants